Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory Part 1. Theoretical foundation

This paper is the first part of a two-part investigation of a novel approach to optimally control commercial building passive and active thermal storage inventory. The proposed building control approach is based on simulated reinforcement learning, which is a hybrid control scheme that combines features of model-based optimal control and model-free learning control. An experimental study was carried out to analyze the performance of a hybrid controller installed in a full-scale laboratory facility. The first part presents an overview of the project with an emphasis on the theoretical foundation. The motivation of the research will be introduced first, followed by a review of past work. A brief introduction of the theory is provided including classic reinforcement learning and its variation, so-called simulated reinforcement learning, which constitutes the basic architecture of the hybrid learning controller. A detailed discussion of the experimental results will be presented in the companion paper. # 2005 Elsevier B.V. All rights reserved.